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Abstract 


A new hybrid variational model is presented for image denoising, which in- 
corporates the merits of Shannon interpolation, total generalized variation 
(TGV) regularization, and a symmetrized derivative regularization term 
based on l'-norm. In this model, the regularization term is a combination 
of a TGV functional and the symmetrized derivative regularization term, 
while the data fidelity term is characterized by the 1?-norm. Unlike most 
variational models that are discretized using a finite-difference scheme, our 
approach in structure is based on Shannon interpolation. Quantitative 
and qualitative assessments of the new model indicate its effectiveness in 
restoration accuracy and staircase effect suppression. Numerical experi- 
ments are carried out using the primal-dual algorithm. Numerous real- 
world examples are conducted to confirm that the newly proposed method 
outperforms several current state-of-the-art numerical methods in terms of 
the peak signal to noise ratio and the structural similarity (SSIM) index. 
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1 Introduction 


Image processing is an extensively utilized and rapidly growing area in com- 
puter science enriched with various applications, including image restoration 
[4], medical visualization [7], industrial inspection [27], law enforcement [50], 
and so on. Recently, various techniques have been proposed in image pro- 
cessing, which aims to recover the underlying true image from its degraded 
observation, including image denoising models [19, 49, 40, 47], image deblur- 
ring models [22, 17], image inpainting models [38, 15], image deconvolution 
models [28, 2], and so on. Image denoising is one of the most fundamental 
problems in image processing and computer vision. In the past few decades, 
many image denoising models have been introduced in the literature, such 
as variational models [39, 45], wavelet transform based models [32], nonlocal 
means filters [48], curvelet transform based models [29], bilateral filters [3], 
and so on. Variational models have been widely used in image denoising 
due to their capability in image feature preservation [33, 21, 43, 42, 44]. TV 
regularization proposed by Rudin, Osher, and Fatemi [30] (the ROF model) 
is a successful example of variational models. Many different papers have 
reported the efficiency of TV regularization [37, 13, 8, 9, 36, 18, 46]. Assume 
that TC R?@ is a continuous domain and that s : T — R is a d-dimensional 
image in continuous domain I. The continuous TV semi-norm of s is defined 
as 


TV(s) = sup { [s divu dx | v €Cl(T,R?), |lulleo < i} =i |Ds| dx, 
r F 


where |-| is the Euclidean norm and |Ds| denotes the variation-measure of 
the distributional derivative Ds, which is a vector-valued Radon measure. 
For smooth function s, we have Ds = Vs. Therefore, TV is the integral of 
its gradient magnitude: 


TV(s) = | ial he. 


In TV regularization, the space of functions of bounded variation (BV) is 
considered as an appropriate class for image restoration. The TV semi-norm 
defines the norm ||s||By := ||s||z1 + TV(s) on the space of BV functions. 
Although TV regularization-based techniques have become one of the most 
widely used regularizers due to their edge-preserving ability, they leave some 
form of staircase artefacts in the smooth regions of the image. To alleviate 
the staircase effect, several high-order methods have been proposed in the 
literature [12, 34, 14, 35, 25]. One technique to overcome this drawback is 
total generalized variation (TGV) regularization [5]. In most algorithms, the 
TV and TGV regularizations are discretized using a finite-differences scheme. 
For this reason, denoised images produced by these approaches cannot be 
efficiently interpolated using standard interpolation models. Several types of 
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research have been conducted as a remedy to this drawback [24, 26]. Shannon 
interpolation has been recently utilized in [1] for total variation-based image 
restoration leading to image improvements in terms of restoration accuracy. 


In the present work, a new hybrid total generalized variation model is 
studied for image denoising in which the regularization term is considered to 
be a combination of TGV regularization and a symmetrized derivative regu- 
larization term based on /!-norm, and the data fidelity term is characterized 
by the /?-norm. Furthermore, in our approach, Shannon interpolation is used 
rather than a finite-differences scheme yielding efficient images in terms of 
restoration accuracy. Quantitative and qualitative evaluations indicate that 
the proposed method significantly outperforms several current state-of-the- 
art methods in staircase effect suppression and restoration accuracy. We 
employ the primal-dual algorithm presented in [11] to provide a global min- 
imizer to the new proposed variational model. 


The outline of the paper is organized as follows. In Section 2, we present 
the necessary definitions and basic properties of TGV model and Shannon 
interpolation. In Section 3, our proposed method and some necessary ana- 
lytical discussions are presented. Section 4 contains the numerical algorithm. 
In Section 5, we present the convergence analysis for the proposed model. 
Section 4 is devoted to numerical experiments. Finally, Section 7 contains 
some concluding remarks. 


2 Preliminaries 


In this section, we first introduce some notations used throughout this pa- 
per, and then we briefly review the concepts of TGV regularization [5] and 
Shannon interpolation [1]. 


2.1 Notations 


1. Q=Iyn, x In, is a discrete domain of size Ni x No, where 
In, = {0,1,2,...,N, — 1}, In, = {0,1,2,...,N2— 1}. 


u € R® is a discrete gray-level image with size N, x No. 
For d>1, I C R¢ is a continuous domain. 

Lj.,.(L) is the set of locally Lipschitz integrable functions s:T > R. 
Xo and A, are fixed positive parameters. 


Sym?(R®) is the vector space of symmetric 2-tensors defined by 


SOs eS ge 


Sym?(R¢) = {n :R? x R? > R | 7, is 2-linear and symmetric}, 
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7. C2(I,Sym?(R®)) is the space of 2-times continuous differentiable func- 
tions with compact support from I to Sym?(R%). 

8. C2((0, Ni] x [0, No], Sym?(R?)) is the space of 2-times continuously differ- 
entiable functions with compact support from [0, Nj] x [0, N2] to Sym?(R?). 


2.2 TGV regularization and Shannon interpolation 


For s € Lj,,.(L), the TGV functional of order 2 with weight \ is defined as 
[5] 


TGV?(s) = sup {| s div? dx | v € C(I, Sym?(R*)), ||v|loo < Ao; 
r 


IIdivulloo < ar}, (1) 


7 us Oui 
where A = (Xo, A1), (divv); oF Br, oF ee an oes 


Moreover, the infinity norms ‘of Vv ad the vector field Ww, satire: w = divu, 
are defined as 


tl = oun { (So ft JP +2 oheuOP)” |xer}, 


i<j 


|tlo0 = sup ic jus?) | xe}. 


In second-order TGV regularization model, the space of bounded generalized 
variation functions of order 2, defined as 


BGV2(P) = {s € LP) | TGV?2*(s) < oo}, 


is considered as an appropriate class for image restoration. The Fenchel dual 
formulation of (2) for d = 2 and u € R%?*2 can be written as [6, 20] 


TGV?A(u) = min AollE(P)Ih + Al|Vu — all, (2) 


pe (R2)N1x No 


where Vu and €(p) stand for the gradient and symmetrized derivative oper- 
ators, respectively, which can be formulated as 


OF Pi 5(O; po + OF pi) 


Vu= [dru ayul’, €(p) = 
° 5 (0; p2 + 9, p1) Oy p2 


where (Oy u)(i,j) = uli + 1,7) — uli, j), (OFu)(i,5) = ulé,j + 1) — uli, 5), 
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(0; pi) (4,5) = pilé, 7) — pili — 1,4), and (O7 p1) (7,9) = pit, 7) — pi, 3 — 1) 
((O; p2) (4,7) and (0; p2)(t,7) can be formulated similarly) are discrete first- 
order derivatives [5]. 

The Shannon interpolate U : R? > R of the discrete image u is defined 
as [1] 


Ni No 


1 2 os os ri ( 22 4 By 
UeM=aa Dl} emlo)en.(8) Aa,8) PMMA, (3) 
A a pa 
2 2 
Mist Net «ak Bl 
where w(a, 8) = S- > u(k, 1) e 2" +83) ig the 2-dimensional discrete 
k=0 1=0 


Fourier transform (DFT) of u, and ey, (a) and ey, (3) are the weighting terms 
[1] formulated as 


_f1/2 if lol = ™, _f1/2 ifs) =, 
En, (a) = { 1 otherwise, €w2(8) = 1 otherwise. 


3 Proposed model 


Let U : R? > R be the Shannon interpolate of u € R® defined in (7). For the 
case d = 2 in (2), we define the second-order Shannon interpolation based 
TGV regularization with weight \ as 


STGV2(u) = sup {| U(a,y) div?V (2, y) dady | Ve 


[0,N1] x [0,No] 


Cz ([0, Ni] x [0, N2],Sym*(R*)), ||V]loo < Ao; I|divV loo < a}, (4) 


where A = (Ao, A1). The integral in (4) is generally difficult to compute but 
it can be easily approximated using the Riemann sum as 


1 
STGV?2*(u) = max {SW div7v)_pm.xn, | v € (R*)™2**2, |Ip||,. < Ao, 
n 


lIdivnvlloo < Av}, (5) 
where n > 2 in practice, div? is the discrete second-order divergence operator 
corresponding to the continuous operator div’, and 

1 


1 . kl... 
72 U, div? v) px xNo = me S- U(-, 7? div2v(k, 1), 
(k,IJEQn 
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in which div? v(k,1) = div’V( a iS and 


“9 
nm nm 


Qn = Inn, X Inn, = {0,1,2,...,2Ny — 1} x {0,1,2,...,nN — 1}. 


In order to compute STGV?(u), we need to obtain its Fenchel dual for- 
mulation. For this purpose, we first introduce the discrete gradient operator 
Vn, the discrete symmetrized gradient operator €,, the discrete first-order 
divergence operator div,, and the discrete second-order divergence opera- 
tor div? in the following. To distinguish between the two cases div,p and 
div,v, where p € (R?7)"™:*"%2 and v € (R*)*%:*42, we use the notations 
Vip :=—divi,p and €*v := —div2,v (* denotes the adjoint operator). As 
a result, div? in (3) will be denoted as div,,,(div2,(v)) from now on. 


Let U : R? -— R be the Shannon interpolate of u € R® defined in 
(7). The continuous gradient operator VU is formulated as VU(a,y) = 


[De(U(2,y)) Dy(U(x,y))]|” [5, 1] in which 


Ny Ne 
201 =z zs Qa ee mi( 224 By 
Da(Wlew)) = ing 2 De ay Sm(@) emal) Blan) NT 
a=— 5) p=— 
6 
-_ 2 (6) 
Dy(U ee) = SB er(a) emg(B) Aa, 8) PME 
y v,Y)) = NN ‘ No EN, (Q@) ENo ula, e a, 
oe (7 
7 


For (k,l) € Qn, the discrete gradient operator V, : RN1*N2 —> (R?)"N1*nNo 
can be derived as [1] 


Vnu(k, 1) = VU( 


3 | 


l 
—). (8) 
nm 
For p = [pi pol’ € (R2)P%1*"N2 and (k,1) € Qn, the discrete first-order 
divergence operator div, : (R?)"™1*"%2 —5 R%1*N2 is defined as 

divin (p(k, 1)) = divin, (p(k, 1) + divin, (p(k, 1)) (9) 


er! eee 


[5], where divin, (p)(a, 8) = Dz(p1)(@, 8) and divin, (p)(a, 8) = Dj(p2)(a., 8), 
for (a, 8) EQ = Iy, x In,, are formulated as [1] 
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Bi(a, 8) if lal < 4B, IA] < B, 
5(Pi(a, 8) — pi(—a, 8)) if a= —-%, |p| < %, 
a 
Dx(p1)(a, 8) = 2ri=— pa a ‘ 
Ni $(pi(a, 8) + pila, —B)) if |a| < MB — —*, 
+ ae Dy, 81Pi (sia, 628) if (a, 8) = (-%, -4B) 
s1= sQo= 
(10) 
pa(a, B) if lal < || < B, 
. 3(P2(a, 8) + B3(-a,8)) if a=—B, |p| < B, 
D* ,B) = 2ni—— 11 
ESE, 1 (G(a8)—la,—A)) if fal < p=, 
1 D _ s2p2(sia,s28) if (a,8) = (-,-4B). 
sy=tlsg=t+1 
Vit Viz ‘i 
For V = € C?([0, Ni] x [0, No], Sym*(R?)), the first-order con- 
Via Vo2 


tinuous divergence operator div2 can be written as [5] 
D, (Vir (a, y)) or D,(Vi2(2, y)) 
D,z(Vi2(a, y)) alr, D,(V29(z, y)) 


where D, and D, are defined as (6) and (7). Now, the discrete first-order 
divergence operator divz,, : (R*)%1*%2 —+ (IR?)"%1*"2 can be derived as 


diveV (2, y) = 


? 


ae), (12) 


nn 
for vy € (R*)§1*%2 and (k,l) € Qn. For p € (R?)"%1*"42 and (a, 8) € 


—_. 


Q = le x Iy,, the symmetrized gradient operator €, : (R?)"41*"%2 —; 
(R*)42*42 is given by [5] 


div2,v(k,l) = diveV( 


a Ds(pi)(a,8) — $(De(p1)(a, 8) + Dz(p2)(a, B)) 
1 (Ds (pi)(a, B) + Dz (p2)(a, B)) Ds (p2)(c, B) 


oe ae 


where D*(p1), D*(po), Ds (p1), and D*(p2) are defined similar to (10) and 
(11). For (k,l) € Qn, Enp(k,!) is obtained by taking the inverse Fourier 


transform as 
En D(k, 1) Eng P(k, 1) 


n k,l => 5 13 
Ny ity esol) ve 
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where &,,, p(k, 1!) is achieved as 


Na Na 
1 “ - Qnri( eh 4 BL) 
n k,l — D*(p Qa, € Na TONS ’ 
ub) = aye LL Peodla.0) 
So =o, 


and €,,p(k, 1), &,,p(k,/) can be formulated similarly. 
The discrete functional STGV2 in (3) is equivalent to Fenchel dual for- 
mulation 


r 
2Af.\ : 70 _ = 
STEVI) = mR og, Galleu@lh+ SllVau—alh, 
where V,, and €, are defined as (8) and (13), respectively. The analytical 
process for establishing this Fenchel dual formulation of (3) is exactly based 


1 
on the lines of [5] and [20] (except the factor — in (3) and the change on 
n 


the operators). For this reason, we do not bring the proof here and we refer 
the reader to the mentioned references. Now, we concentrate on the new 
Shannon interpolation based the hybrid TGV (SHTGV) model defined as 


SHTGV;*(u) = STGV2*(u) +7 [lén(Vn(u))lla, 


where y > 0 is a fixed positive parameter, STGV?(u) is defined as (14), 
and &,(Vn(u)) can be easily formulated by substituting Vn(u) with p in 
the procedure of deriving €, in (13). The concerned variational problem is 
established in the form 


min ||u — uo||3 + SHTGV;*(u), (15) 


where u is the desired restored image and uo is the observed data. According 
to our experience, the fixed value y = 0.01 is suitable and does not need to 
be tuned. Therefore, one can say that no additional parameter is introduced 
with the new model. 


4 Numerical algorithm 


We use the primal-dual algorithm (Algorithm 1) presented in [11] to provide 
a minimizer of the variational problem (9). This primal-dual algorithm is 
particularly efficient in finding a solution to the saddle-point problem 


min mas (Ka,y)y + F(x) — Gly), (16) 
where X and Y are Hilbert spaces, kK : X —> Y is a linear and continuous 
mapping, and F : X —+ (—co,oo] and G : Y —+ (—oo, co] are proper, 
convex, and lower semi-continuous functionals. In order to adapt the opti- 
mization problem (9) to Algorithm 1, we need the corresponding saddle-point 
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Algorithm 1 The first-order primal-dual algorithm presented in [11]. 
1. Set k = 0, choose the initial estimates 7° € X, y®° € Y, #9 = x°, and 
positive parameters tT and o such that ot|||A'|||? < 1 (||| - ||| denotes the 
induced [?-norm). 
2. Calculate x**!, y**+1, and z*+! using the following equations: 

y*t! = (I +0 0G) 1(y* +0 K z*), 

okt! = (I+7 OF) 1 (a* —7 K* y**), 

ghtl — okt] _ pk 
3. Stop or set k = k+1 and go back to step 2. 


structure of this optimization problem, which is formulated as (see for details) 


At Ao 
. 2 ee = pals 
etna gee ae Iu — wolla + 3 (Vine — p,w) + =3 (En(p),¥) 

pE(R2)rN1xrNo vE(R4)N1xNe2 

qe (R4) Nix No 
+ VEn(Vnu), 4) — Leic<iy(@) — Leics) = Littlecsay(@)- 
(17 
Using the saddle-point structure (17), x, y, the functionals F' and G, and the 
mapping K in saddle-point problem (16) are obtained as 
v= (u, P), y= (w,y, q); F(z) = F(u,p) = ||u — uoll5; 


Gy) = Gw,v, 9) = Lg o<n() +L cs + Lgitc<1}(M> 


a By 
— Xr 
- 00 Se 
YEnVn 0 


From [1, Proposition 7], we have |||V ||| < 7/2. Moreover, some com- 
putations yield |||€,,||| < naV2. Therefore, an upper bound for the induced 
[?-norm of K can be easily achieved as 
2m? (Ag)? + (8)2)n2 + dny?n! + (28)? 


KP? < 
|||] 5 
(Cay + (A)?)n? + 4nty?n4 + (A4)?)? 
7 A 
= —16m4n4((Ab)? (Ag)? + (Ad)?y7) — 32%? (A8)?n8\ 1/2 


i ; 
Now the primal-dual algorithm for solving (9) can be formulated as (see [11] 
for more details), 
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A i 
uk} = arg min{||u—uoll3 + (Vat p,w) +V(En(Vnts), 4) + 5—llu— uw I3} 


ML. ; : 
uk + 7 divin (w**?) - Tydivin (dive, (q**")) +2ru? 


— 1 
1+ 27 , ( 8) 
k+1 Al Xo 1 2 
p' =argmin{—;(Vnu— p,w) + 5 (En(p),¥) + =—llp — P'llat 
D n n 2T 
r ro .. 
=p*+r Co wrth + “piven (v***)), (19) 
k4+1 Ay 1 ky2 
wr’ = argmax{—5(Vnu — p,w) — Iy)4),.<1}(w) — s-llw — w" la} 
w mr 20 
Xr 
= Py,(w* + a (Vna* — p*)), (20) 
k+1 Xo 1 ky2 
ers arg max{—> (én(p),v) = Tadic) — 3, — Vv" |l2} 
A = 
= Pr(v¥ +03 (E(6"))), (21) 
. 1 
ght! = arg max{y(En(Vnu), 9) — Lqy.0<13(@) — 3 lI9 — qh ||3} 
q 
= Py(g* + oy (En(Vnt*))), (22) 
where 
_ w _ 7 7 q 
= Tati Tay = maxi’ 7) = max Ta’ 


and 7 and a are positive constants. Algorithms 2 is the adapted algorithm 
to solve (9) (see [11] for details). 
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Algorithm 2 The adapted algorithm for solving the SHTGV model. 
1. Set &k = 0, choose the initial estimates u2 € R142, p® € 
(ie ee, w? E (Re?) ane, yp? E (IR*)N2 Na, g° E (IR*)M1%* Na, 
u° = u°, p°? = p®, and positive parameters 7, o such that o7|||.K|||? < 1, 
where 

2 

2 
ATI < 2 
(ono 2)? +4n4y?n4+(Ab)?)? 

7 4 
1604m4(( A$)? (29 )?-+( A$ )?77) 3207? (29 )?n8 \ 1/2 

4 
2. Calculate u®+1, pet}, wet, pe+1, q+] ak+1 and p*+! using the following 
equations: 


r 
wrtt = Py (we + oS (Vn — p*)), where Py, (w) = 


(AR)? 4 (29)? m2 +4 ty?2nt (Ab)? 


Xv 7 
k+1 _ k A0 ak =) — 
V =Py,(v wee (En(D )))s where P), (7) max(1, |7|)’ 
. _ = qd 
il = Pa + 07(En(Vnit*))), where P,(q) = max(1,|4l)’ 
uk + 7 divin (wht) - Tydiv,,(div2,,(q*t")) +27ru? 
yeti = n 


Xr r 

ot 1 et 0 4: + 

pe 1 p* T( wr 1 div le ae 
qkt = Qu — uf, pe x => 2p* 1 


k 
— pk, 
3. Stop or set k = k+1 and go back to step 2. 


5 Convergence analysis 


In this section, we establish the convergence of the primal-dual algorithm 
used for solving (9). Here, we only give the basic proof frameworks and do 
not repeat the lengthy proving process in [11]. For this purpose, we first 
prove the existence of the solution to the variational problem (9) through the 
following theorem, which proof is similar to the proof of Theorem 1 in [23]. 


Theorem 1. There exists a solution to optimization problem (9). 


Proof. Using the compactness property of BV(Q), for every bounded and 
minimizing sequence {ux }ren C BV(Q), there exists a subsequence {ux, }ien C 
{ux}ren converging to some u* € BV(Q). According to [5], the functions 
|Vnu — pila, [En(p)|l4, and |/&n(Vn(u))||1 are proper, convex, and lower 
semi-continuous, and the same conclusion holds for their weighted sum 
SHTGV2(u). Moreover, {||Augz; — uol|Z}ien is bounded. Therefore, we 
yield 

|| Au* — uo||3 + SHTGV2(u*) < lim inf (|| Aug, — uo||3 + SHTGV2 (ug, )). 


i-+0o 
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As a result, we can conclude the existence of the solution to optimization 
problem (9). 


Now the convergence theorem is exhibited in the following result. 


Theorem 2. For positive parameters ¢ and 7, where o7|||K|||? < 1, the 
sequence {u*, p*, w*, v*, q*} generated by Algorithm 2 converges to a solution 
of (9). 


Proof. Based on Theorem 1, there exists a solution to optimization problem 
(9). For this reason, the set of saddle-points of (17) is non-empty. According 
to [5], the functions ||Vnu — pla, |[En(p)||1, and ||E&:(Vn(w))||1 are proper, 
convex, and lower semi-continuous functions. Moreover, o and 7 in Algorithm 
2 satisfy o7|||K|||? <1. As a result, conditions (a), (b), and (c) of Theorem 
1 in [11] hold. Therefore, we can conclude that Algorithm 2 is an application 
of the original primal-dual algorithm in [11] and converges to a solution of 


(9). 


6 Experimental results 


We compare the experimental results of our model with the current state-of- 
the-art models: isotropic TV (ITV), upwind TV [10] (UTV), Condat TV [16] 
(CTV), TGV [5], Shannon TV [1] (STV), and the Huber variant of Shannon 
TV [1] (HSTV). The experiments are conducted on test images illustrated 
in Figure 1. All test images are corrupted by additive white Gaussian noise 
of standard deviations 0.18 and 0.23. All the reconstruction processes are 
carried out using MATLAB R2015b on a PC with AMD A10-4600M APU 
with Radeon(tm) HD Graphics, 2.3-GHz Inter Core processor, and 4GB of 
RAM under Windows 8. 

As discussed in [1], in practice, choosing the oversampling factor as n = 2 
or n = 3is enough. We set the oversampling factor n = 2 in STV, HSTV, and 
the new proposed SHTGV model. For the case n = 3 in STV, HSTV, and 
the new proposed SHTGV model, we observed that the convergence speed is 
decreased while no remarkable change is made in comparison with the case 
n= 2. 

The restoration quality is measured by the peak signal to noise ratio 
(PSNR) [31] and the structural similarity index measure index [41]. The 
best result for each comparison item is highlighted in bold type font (see 
Table 2). The restored results are illustrated in Figures 2, 3, 4, 5, 6, and 7. 
For each model and test image, the results corresponding to parameters that 
provide the highest PSNR value are presented here. 

The results of the ITV, UTV, CTV, STV, and HSTV models obviously 
illustrate staircasing effect in the affine regions. Although the TGV model 
is capable of suppressing the staircasing artifacts, the denoised results using 
this approach indicate that it sometimes leads to undesired edge blurring. 
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Moreover, the performance of the TGV model reveals that some regions con- 
tain an obvious staircasing effect. Our proposed model not only substantially 
alleviates the staircasing artifacts, but also sharply preserves important im- 
age features. Moreover, the reconstructed edges and details by our model 
are more distinct in comparison with other variational models. For example, 
zoomed-in regions in Figures 2 and 3 illustrate that the UTV model leaves 
some small white noise particles in denoised results. The ITV model performs 
to some extent better than the UTV model in noise removal, but some details 
and edges are not well reconstructed during the denoising process. The CTV 
model outperforms both the UTV and ITV models in restoration accuracy 
and noise removal. However, the results of the ITV, UTV, and CTV models 
obviously suffer from the staircasing artifacts. Although the STV and HSTV 
models have better performance in preserving edges and details in compari- 
son with the ITV, UTV, and CTV models, they achieve results with obvious 
staircasing effect, while the staircasing effect is less sharp compared with the 
ITV, UTV, and CTV models. The denoised results by the TGV model in- 
dicate that this regularization scheme sometimes produces denoised results 
with blurred edges. The denoised results using the proposed model illustrate 
that this new scheme is capable of eliminating the staircasing effect and pre- 
serving the edges at the same time. In fact, the ITV, UTV, CTV, STV, and 
HSTV models cannot compete with our model in staircasing effect suppres- 
sion, as observed in the denoised results of Figure 2. TGV model is capable 
of eliminating the staircasing effect to some extent, but it yields denoised 
results with blurred edges. As a result, we can conclude that TGV model 
cannot compete with our model in edge preservation. The same conclusion 
holds for denoised results in Figures 4, 5, and 6. 


The ITV, UTV, CTV, and TGV regularization models are discretized 
using a finite-difference scheme, while the STV, HSTV, and SHTGV mod- 
els are based on the concept of Shannon interpolation. For this reason, the 
images generated by means of ITV, UTV, CTV, and TGV schemes cannot 
be efficiently interpolated using standard interpolation models. To support 
this claim, croppings of the restored results are magnified with factor 3 using 
bicubic interpolation method, which reveals that interpolating the restored 
results of the ITV, UTV, CTV, and TGV models yield images with unwanted 
artifacts, while the results of the STV, HSTV, and SHTGV models are in- 
terpolated without creating artifacts. Since the proposed model outperforms 
both STV and HSTV models in staircase artifacts suppression, the staircas- 
ing artifacts are remarkably alleviated in interpolated results of our model. 
Moreover, the edges and details in interpolated results of our model are more 
distinct in comparison with the STV and HSTV models. 


IJNAO, Vol. 12, No. 2, pp 371-396 


384 Tavakkol, Hosseini and Hosseini 


Figure 1: The test images used in our numerical experiments. First row, from left to 
right: Bird, Parrot, and Lena. Second row, from left to right: Clown, MRI1, and MRI2. 
Third row, from left to right: MRI3, MRI4, and MRI5. Forth row, from left to right: 
MRI6 and MRI7. 
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Figure 2: Performance comparison of Lena test image with additive white Gaussian 
noise of standard deviation 0.18. First row, from left to right: reference image, noisy 
image, restored results by ITV, UTV, and CTV. Second row, from left to right: restored 
results by STV, HSTV, TGV, and SHTGV. Images in the third and fourth rows are the 
corresponding zoomed-in regions of images in the first and second rows, respectively. 
Images in the fifth and sixth rows are croppings of the restored results of the Lena test 
image in the first and second rows magnified with factor 3 using the bicubic interpolation 
method. Fifth row, from left to right: reference image, resampling of the reference image, 
ITV, UTV, and CTV. Sixth row, from left to right: resampling of and STV, HSTV, 
TGV, and SHTGV. 
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Figure 3: Performance comparison of MRI1 test image with additive white Gaussian 
noise of standard deviation 0.18. First row, from left to right: reference image, noisy 
image, restored results by ITV, UTV, and CTV. Second row, from left to right: restored 
results by STV, HSTV, TGV, and SHTGV. Images in the third and fourth rows are the 
corresponding zoomed-in regions of images in the first and second rows, respectively. 
Images in the fifth and sixth rows are croppings of the restored results of MRI1 test 
image in the first and second rows magnified with factor 3 using bicubic interpolation 
method. Fifth row, from left to right: reference image, resampling of the reference 
image, ITV, UTV, and CTV. Sixth row, from left to right: resampling of and STV, 
HSTV, TGV, and SHTGV. 
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Figure 4: Performance comparison of MRI2 test image with additive white Gaussian 
noise of standard deviation 0.18. First row, from left to right: reference image, noisy 
image, restored results by ITV, UTV, and CTV. Second row, from left to right: restored 
results by STV, HSTV, TGV, and SHTGV. Images in the third and fourth rows are the 
corresponding zoomed-in regions of images in the first and second rows, respectively. 
Images in the fifth and sixth rows are croppings of the restored results of MRI2 test 
image in the first and second rows magnified with factor 3 using bicubic interpolation 
method. Fifth row, from left to right: reference image, resampling of the reference 
image, ITV, UTV, and CTV. Sixth row, from left to right: resampling of and STV, 
HSTV, TGV, and SHTGV. 
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Figure 5: Performance comparison of Bird test image with additive white Gaussian 
noise of standard deviation 0.18. First row, from left to right: reference image, noisy 
image, restored results by ITV, UTV, and CTV. Second row, from left to right: restored 
results by STV, HSTV, TGV, and SHTGV. Images in the third and fourth rows are 
the corresponding zoomed-in regions of images in the first and second rows, respectively. 
Images in the fifth and sixth rows are croppings of the restored results of Bird test image 
in the first and second rows magnified with factor 3 using bicubic interpolation method. 
Fifth row, from left to right: reference image, resampling of the reference image, ITV, 
UTV, and CTV. Sixth row, from left to right: resampling of and STV, HSTV, TGV, 
and SHTGV. 
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Figure 6: Performance comparison of Parrot test image with additive white Gaussian 
noise of standard deviation 0.18. First row, from left to right: reference image, noisy 
image, restored results by ITV, UTV, and CTV. Second row, from left to right: restored 
results by STV, HSTV, TGV, and SHTGV. Images in the third and fourth rows are the 
corresponding zoomed-in regions of images in the first and second rows, respectively. 
Images in the fifth and sixth rows are croppings of the restored results of Parrot test 
image in the first and second rows magnified with factor 3 using bicubic interpolation 
method. Fifth row, from left to right: reference image, resampling of the reference 
image, ITV, UTV, and CTV. Sixth row, from left to right: resampling of and STV, 
HSTV, TGV, and SHTGV. 
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Table 1: Comparison of different models in terms of PSNR and SSIM with additive 
white Gaussian noise of standard deviation 0.18. 


Image MRI MRI2 
Model PSNR / SSIM / Optimal PSNR / SSIM / Optimal 
ITV 25.30 / 0.7810 / 0.15 25.33 / 0.7687 / 0.15 
UTV 24.76 / 0.7566 / 0.18 24.80 / 0.7463 / 0.18 
CTV 25.64 / 0.7898 / 0.14 25.64 / 0.7792 / 0.14 
STV 26.14 / 0.8113 / 0.25 26.06 / 0.7996 / 0.25 
HSTV 26.07 / 0.7965 / 0.25 26.00 / 0.7936 / 0.26 
TGV 26.01 / 0.7824 / (0.4, 0.1) 26.10 / 0.7759 / (0.5, 0.1) 
SHTGV | 26.39 / 0.8210 / (0.45,0.28) | 26.34 / 0.8139 / (0.45, 0.28) 
Image Bird Parrot 
Model PSNR / SSIM / Optimal PSNR / SSIM / Optimal 
ITV 25.50 / 0.7348 / 0.15 25.33 / 0.7428 / 0.15 
UTV 25.03 / 0.7109 / 0.18 24.80 / 0.7127 / 0.18 
CTV 25.78 / 0.7438 / 0.14 25.63 / 0.7522 / 0.14 
STV 26.09 / 0.7563 / 0.25 25.73 / 0.7464 / 0.24 
HSTV 26.04 / 0.7509 / 0.26 25.68 / 0.7349 / 0.24 
TGV 26.03 / 0.7416 / (0.17, 0.11) 25.46 / 0.7584 / (0.15, 0.29) 
SHTGV | 26.29 / 0.7693 / (0.45,0.25) | 25.84 / 0.7620 / (0.42, 0.21) 


Table 2: Comparison of different models in terms of PSNR and SSIM with additive 


white Gaussian noise of standard deviation 0.23. 


Image MRIL Parrot 
Model | PSNR / SSIM / Optimal A | PSNR / SSIM / Optimal 
ITV 24.10 / 0.7480 / 0.2 24.17 / 0.7111 / 0.2 
UTV 23.55 / 0.7053 / 0.23 23.63 / 0.6603 / 0.23 
CTV 24.42 / 0.7454 / 0.18 24.46 / 0.7086 / 0.18 
STV 24.90 / 0.7762 / 0.33 24.56 / 0.7146 / 0.32 
HSTV 24.84 / 0.7608 / 0.33 24.51 / 0.7020 / 0.32 
TGV 24.75 / 0.7439 / (0.26,0.14) | 24.30 / 0.7117 / (0.19, 0.36) 
SHTGV | 25.03 / 0.7551 / (0.5,0.5) | 24.62 / 0.7178 / (0.4, 0.28) 
Image Lena Clown 
Model | PSNR / SSIM / Optimal A | PSNR / SSIM / Optimal 
ITV 24.49 / 0.6701 / 0.21 22.42 / 0.6308 / 0.19 
UTV 24.03 / 0.6393 / 0.25 21.97 / 0.6035 / 0.22 
CTV 24.67 / 0.6759 / 0.19 22.67 / 0.6443 / 0.18 
STV 24.69 / 0.6818 / 0.33 22.89 / 0.6563 / 0.31 
HSTV 24.68 / 0.6814 / 0.34 22.89 / 0.6533 / 0.31 
TGV 24.77 / 0.6916 / (0.21,0.17) | 23.00 / 0.6471 / (0.23, 0.13) 
SHTGV | 24.83 / 0.6926 / (0.5,0.3) | 23.10 / 0.6645 / (0.6, 0.37) 
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Table 3: Comparison of different models in terms of PSNR and SSIM with additive 
white Gaussian noise of standard deviation 0.18. 


Image MRI3 MRI4 
Model PSNR / SSIM / Optimal PSNR / SSIM / Optimal 
ITV 24.57 / 0.7567 / 0.15 25.60 / 0.6891 / 0.15 
UTV 24.06 / 0.7354 / 0.18 25.17 / 0.6676 / 0.18 
CTV 24.91 / 0.7671 / 0.14 25.83 / 0.6966 / 0.14 
STV 25.28 / 0.7810 / 0.25 26.16 / 0.7161 / 0.25 
HSTV 25.25 / 0.7739 / 0.25 26.11 / 0.7096 / 0.26 
TGV 24.99 / 0.7470 / (0.29, 0.09) 26.11 / 0.6988 / (0.16, 0.12) 
SHTGV 25.44 / 0.7893 / (0.4, 0.2) 26.38 / 0.7231 / (0.4, 0.2) 
Image MRI5 MRI6 
Model PSNR / SSIM / Optimal PSNR / SSIM / Optimal 
ITV 25.96 / 0.6351 / 0.16 23.30 / 0.6245 / 0.14 
UTV 25.50 / 0.6173 / 0.19 22.86 / 0.6057 / 0.16 
CTV 26.19 / 0.6413 / 0.15 23.58 / 0.6288 / 0.13 
STV 26.43 / 0.6541 / 0.26 23.89 / 0.6285 / 0.22 
HSTV 26.39 / 0.6429 / 0.26 23.86 / 0.6178 / 0.22 
TGV 26.22 / 0.6174 / (0.17, 0.11) 23.67 / 0.5766 / (0.24, 0.08) 
SHTGV 26.53 / 0.6444 / (0.4, 0.2) 24.06 / 0.6291 / (0.4, 0.2) 
Image Lena Clown 
Model PSNR / SSIM / Optimal PSNR / SSIM / Optimal 
ITV 25.45 / 0.7028 / 0.15 23.55 / 0.6866 / 0.14 
UTV 25.02 / 0.6800 / 0.19 23.08 / 0.6587 / 0.16 
CTV 25.67 / 0.7133 / 0.14 23.82 / 0.6979 / 0.13 
STV 25.70 / 0.7189 / 0.25 23.99 / 0.7083 / 0.23 
HSTV 25.69 / 0.7160 / 0.25 23.99 / 0.7053 / 0.23 
TGV 25.77 / 0.7277 / (0.17, 0.11) 24.16 / 0.7028 / (0.18, 0.09) 
SHTGV | 25.84 / 0.7283 / ((0.42,0.23) | 24.24 / 0.7149 / (0.44, 0.25) 
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Figure 7: Performance comparison of denoised results of MRI7 with additive white 
Gaussian noise of standard deviation 0.18. First row, from left to right: reference image, 
noisy image, restored results by ITV (optimal A = 0.17, PSNR = 25.63, SSIM = 0.7555), 
UTV (optimal \ = 0.16, PSNR = 25.79, SSIM = 0.7639), and CTV (optimal \ = 0.17, 
PSNR = 25.63, SSIM = 0.7555). Second row, from left to right: restored results by 
STV (optimal \ = 0.26, PSNR = 25.87, SSIM = 0.7774), HSTV (optimal A = 0.26, 
PSNR = 25.81, SSIM = 0.7507), TGV (optimal \ = (0.17,0.11), PSNR = 25.84, SSIM 
= 0.7113), and SHTGV (optimal \ = (0.46, 0.22), PSNR = 25.99, SSIM = 0.7525). 


7 Conclusion 


In this paper, a new hybrid TGV model was presented for image denoising. 
This model incorporated the advantages of Shannon interpolation, TGV reg- 
ularization, and a symmetrized derivative-based ['-norm regularized term. 
In comparison with some state-of-the-art techniques, experimental results 
illustrated that the proposed model substantially alleviates the staircasing 
effect and sharply preserves important image features. In fact, the recon- 
structed edges and details by our model were more distinct in comparison 
with several current state-of-the-art variational models. As we mentioned in 
the paper, most variational models were discretized using a finite-difference 
scheme, which cannot be efficiently interpolated using standard interpola- 
tion models. In this paper, we observed the efficiency of applying Shannon 
interpolation instead of the finite-differences schemes in the structure of vari- 
ational models. Shannon interpolation applied in this paper was based on 
the Fourier transform. Designing a more powerful structure for Shannon in- 
terpolation based on the curvelet transform instead of Fourier transform will 
be considered as our future research. 
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